GeoRep: A Flexible Tool for Spatial Representation of Line Drawings

A key problem in diagrammatic reasoning is understanding how people reason about qualitative relationships in diagrams. We claim that progress in diagrammatic reasoning is slowed by two problems: (1) researchers tend to start from scratch, creating new spatial reasoners for each new problem area, and (2) constraints from human visual processing are rarely considered. To address these problems, we created GeoRep, a spatial reasoning engine that generates qualitative spatial descriptions from line drawings. GeoRep has been successfully used in several research projects, including cognitive simulation studies of human vision. In this paper, we outline GeoRep’s architecture, explain the domainindependent and domain-specific aspects of its processing, and motivate the representations it produces. We then survey how GeoRep has been used in three different projects–a model of symmetry, a model of understanding juxtaposition diagrams of physical situations, and a system for reasoning about military courses of action. Introduction: How Diagrams Work Diagrams are ubiquitous. In daily communications, through sketches, maps, and figures, people use diagrams to convey information. Some diagrams depict intrinsically spatial domains, such as bus routes or furniture arrangements. Other diagrams use spatial concepts to compactly show more abstract relations, such as corporate hierarchies or data flow in a computer program. In all such domains, diagrams can be extremely effective. It is also true, however, that there is a keen difference between effective and ineffective diagrams. Small visual differences may distinguish a diagram that elucidates from one that confuses (Tufte, 1990). A key difference between good and bad diagrams is how well they utilize the kinds of qualitative spatial relations most easily perceived by the human visual system. In the best diagrams, these spatial relations support the conceptual relations the reader is meant to infer. For example, in a thermodynamics diagram, an arrow may indicate the direction of heat flow, with thicker arrows to indicate greater flow, or tapering arrows to indicate heat dissipation. Or, in a circuit diagram, wires Copyright © 2000, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. may be drawn so that related wires are adjacent and parallel, so they can be visually grouped. For this reason, to understand how diagrams work, we must show how diagrams use visual characteristics to support particular qualitative inferences. In the system described here, we model this process as an interaction between two representation levels: 1. A low-level, domain-independent representation which involves a representative set of primitive spatial relations. This level models human low-level vision. 2. A high-level, domain-specific representation that models visual skills for a particular domain. This level links lowlevel visual relations to a domain’s conceptual content. These two representation levels form the basis of GeoRep. GeoRep is an engine for building diagrammatic reasoners. GeoRep takes as input a line drawing, given as a set of primitive visual elements. From this drawing, GeoRep creates a predicate calculus representation of the drawing's visual relations. To perform this task, GeoRep, given the drawing, examines the primitive shapes in the figure, looking for a broad set of low-level visual relations. These relations are detected by a library of visual operations (assumed to be domain-independent) which partially cover the set of universal visual routines (Ullman, 1984). Next, GeoRep uses these relations, in combination with domain-dependent rules, to generate the second, domain-specific representation. GeoRep's two-level